Computer graphic operations that are initiated by pointing devices are typically two types: drawing operations and control operations. Drawing operations describe loci for the placement of visible marks on the image, analogous to drawing on a piece of paper by making a motion with a pen in contact with the surface. Control operations are used to initiate and guide the execution of computer functions leading to modification of the visible marks. Control operations are especially important to the performance of editing functions such as erasing, moving, or changing the visual qualities (e.g. line width) of visible marks of the image.
With most user interface systems, control functions are generally instituted by giving commands to the system (e.g. keyboard command languages, menuing techniques, etc.). Giving commands can be simplified into two parts: specifying (creating or selecting) the command and issuing it, i.e., telling the system to "execute it." In a gesture based input system such as the Liveboard (trademark of Xerox Corporation), such control functions can be instituted by either a menu or a command gesture. It is noted that a gesture like other freehand data strokes entered on the touch panel is also a handrawn stroke but it is interpreted as a command designating an action for the system to perform.
With menu functions, a visible menu is either located on a specified area of a display or appears near the stylus position when desired (pop-up menu). In order to institute a simple command from a menu, the user moves to a desired item and selects it by pressing or lifting a button on the pointing device if one were present on the device or by some other means if an ordinary pen were used. If the user does not wish to commit to a control command operation, i.e., issue any item on the menu, the user can move the pointing device off of the menu before issuing the command or move the stylus to a "cancel" selection and lift the stylus from the display.
On the other hand, in order to institute a simple control command by gesture, the user would draw a gesture that represents some desired operation and then commit to this command by lifting the input device (e.g. stylus) or a button on the device (e.g. mouse). Whether issuing commands by menu or by gesture, it is important to note the distinction between selection commands and command operations in the graphical interfaces described above with respect to their location on a touch panel. With graphical interfaces, graphic images are used to represent objects in the system (i.e., files, text, processes, etc.). Commands generally consist of the selection of a graphic object and the application of an operation to the object. For example, in a system that uses menus, the pointing device is used to select an object by pointing to it, a menu is displayed "on top" of the object and the user selects an operation from the menu. In this way operations are applied to objects. Gestures work in a similar manner. The user selects an object then draws a gesture to specify an operation to be applied to the object. In many systems selection and gesture can be combined by drawing the gesture "on top" of the object.
In these graphical interfaces systems, a command may be issued via menu or gesture by direct-touch input or by indirect input. Direct-touch input for a graphic user interface occurs when the user physically touches the input device onto the display. Thus, the physical place of input (the device) is the same as the physical place of output (the display). An example of direct touch input is in the Liveboard where the Liveboard pen touches the liveboard display. When the Liveboard is simulating the whiteboard, the display pixels turn black (or whatever the "pen color" is) at the locations where the pen touches the board, much like "ink" flows from a pen. Of course it is possible for the Liveboard to blacken pixels that are not where the pen is touching, but the effect of doing so is often unnatural, even disturbing for the user.
Indirect input occurs when the input device is moved on a different surface from the display surface. One example of indirect input is a mouse which is moved over a desktop or mouse pad. Another example is the stylus used with an input tablet without a display. In the first example, the relative movements are sent to the computer which computes an effect on the display (usually the movement of a cursor). In the second example, the points of the tablet are given corresponding points on the display. These are usually called relative position and absolute position devices.
Although direct-touch is very natural for a user there are some disadvantages. Sometimes it is desirable for a system to translate the cursor's position from a point where it is naturally located, given the position of the input device. Consider for example the case of the pop-up menus. If a user points to the center region of a display, the pop-up menu appears at the cursor's location. The user then moves the input device to select an item from the menu. This works for both direct touch and indirect input. However, if a user points into a corner region on the display, the entire menu does not fit on the display.
With indirect input using a relative position device, this can be accomplished easily by placing the pop-menu towards the center of the display where it will fit and translating the cursor to be in the menu. With direct touch, it is not possible to translate the cursor, since the cursor is by definition always "under the pointing device," i.e., the lo pen tip or finger tip. Therefore, while the menu can be moved to a better location by the system, the user must physically move the input device. A similar problem exists with gestures. If there is :not enough room to draw a gesture, with indirect input, the cursor can be translated to a new position where there is enough room to draw and drawing can continue. This cannot be done with direct touch.
It would be desirable to have the same advantage as indirect input available to those direct-touch input devices.